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ABSTRACT 

Because information flow is considered essential to 
an organization and because decision makers are dependent upon 
accurate information* the distortion of messages as they move upward 
in an organization has become a central concern of many 
organizational communication researchers. The purposes of this paper 
are to critically review the various approaches to operationalizing 
such ,f upward distortion," to emphasize those which are most 
misleading, and to suggest operational procedures which will provide 
more valid indices of upward distortion* The following four 
categories of operational definitions purporting to measure upward 
distortion are reviewed: disparity scores, questionnaire/interview 
data, actual encoding of messages, and selection/transmission of 
messages, (JfU 
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ABSTRACT 

This paper posits that many of the conclusions concerning uptmrd distortion 
In organizational hierarchies gre based upon operational procedures which 
are equivocal and often lead to confounding- interpretations. This thesis 
was supported by reviewing four categories of operational definitions pur- 
porting to measure fT up:mrd distortion," . Tho four categories, were: disparity 
scores, questionnaire/interview data, actual encoding of messmges, and 
mleotion/transmission of messages. Following this critical review pro- 
cedures are suggested designed to reduce the distortion in upward distortion 
data* 
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A theme common to mar.'j organizational theorists and researchers 
is that information flow is the "lifeblood" of an organization* In 
order for decision makers to arrive at the best possible decision 
selected from a set of alternative decisions, accurate information 
Is a necessity* Given this theme and its underlying rationale, we 
oan easily understand why upward distortion of organizational messages 
has become a central concern of many organizational communication 
researchers^ 

The centrality of upward distortion as an issue in organizational 
coiittiiunica tlon necessitates that we critically assess the procedures 
used to investigate this concept, This assertion is based upon the 
rationale that "upward distortion** has generated many different 
operational definitions all purporting- to measure the same thing* 
Furthermore^ it appears that many of the findings related to upward 
distortion arm based upon operational definitloiw which are botlh 
equivocal and misleading* Hence, the purpose of this paper is 
twofold: (1) To critically review the various approaches to opera* 
tionalizing "upward distortion" highlighting those which are most 
wdgle&dingf and (2) To suggest operational procedures which will 
provide us with more valid indices of upward distortion* 

Upward Distortion s A Critical Review 
Qf Operational Definitions 

The operational definitions reviewed in this section will be 
subsumed under four general headingsi disparity scores, questionnaire/ 
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interview data, selection/transmission of messages and actual 
encoding of messages. Representative studies will be presented 
under each heading along with a critical assessment of the specific 
Operational definition used in the studies. 

Pis pari ty Scores 

Representative studies using disparity scores as an index of 
upward distortion include those conducted by Read, Mellinger f 
Maier, Hoffman and Read, 3 *?nd Athanassiades. 4 The logic upon which 
the disparity score is based is as follows, If the subordinate 
sent accurate messages to his supervisor , then the supervisor should 
know certain things about the subordinate's organizational space, 5 
Hence, tlie greater the disparity between a supervisor's and sub- 
ordinate's evaluation (rating, ran}:ing) of elements in the subor- 
dinate's space, the less accurate the upward communication . Phrased 
differently , accuracy of upward communication results in accurate 
information at the supervisory level which in turn results in 
congruence between a supervisor's and subordinate's evaluations* 
This logic served as the basis for the specific operational pro- 
cedures adopted by Read, Mellinger, and Maier, Hoffman f and Read, 
Athanassiades' studies, although also utilizing a disparity score f 
was based upon a different rationale and henced utilized a different 
methodology s 

Athanassiades argued that distortion of upward communication 
is "the difference between the index of upward communi cation as it 
occurs and the index of upward communication as it would occur if 
if the subordinate were not distorting. In terms ct specific 
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procedures, Athanassiades asked subordinates to complete Gordon's 

7 $ 
Personal Profile and Inventory Sarnies as measures of self descrip- 
tion and were informed thai the results would be kepi confidential, 
Be then asked subordinates to rate themselves again on the same 

scales r which according to instructions , would be "counter-evaluated 

9 

by their respective superiors Thus, Athanassiades reasoned that 
the former scores represented accurate comr^uni cation , the latter 
scores represented distorted communication, and the disparity between 
the Wo scores represented the degree of upward distortion, 

On the face of it, there does appear to be an intuitive appeal 
to the disparity approach to opera tionali zing "upward distortion*" 
After all, if a subordinate did send accurate messages to his super- 
visor, then the supervisor should know more about the elements in 
the subordinate* s organizational space than if less accurate messages 
were sent* Thus, the degree of agreement within a given supervisor- 
subordinate dyad with respect to the evaluation of elements in that 
space should serve as an index of upward distortion* Unfortunately , 
the appeal of this operational definition is greatly diminished as 
one begins to critically examine both the potentially confounding 
interpretations generated by the procedure and the apparent lack 
of isomorphism between the operational definition and its allied 
theoreticAl construct. The following discussion presents this 
critical examination with the specific reference to £L Read, 
Mellinger , and Maier, Hoffman, and Read studies* Since zhanass iados 9 
studies represent a deviation from the approach of the above three 
studies, his will b# analysed separately* 
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In essence. Read, Mellinger, and Maier , Hoffman, and Read would 
have us believe that because disparity exists in the respective 
evaluations of a superidsor and subordinate, then the latter has 
received distorted information from the former* A more direct and 
parsimonious interpretation, however, is that disparity exists in 
the dyadic evaluations simply because hierarchical differences 
lead to disparity in perceptions , In other words, these three 
studies collected data based primarily upon perceptual processes , 
Supervisors were asked to evaluate various elements on the basis of 
what these elements would "mean" to toe subordinate. Hence, rather 
than conclude that disparity in evaluations is a result of distorted 
upward communication, we can just as easily, and possibly more 
justifiably conclude that the disparity in evaluations is a result 
of differential perceptions associated with different levels of the 
hierarchy, This latter interpretation gains further credence by 
simply considering the extensive body of literature supporting the 
proposition that one's position in an organization shapes one's 
perceptions of the organization*^® 

Secondly, the disparity score may be interpreted in terms of 
encoded distortion, or it may be interpreted in terms o£ decoded 
distortion* That is, the disparity score may also lead to the 
conflicting interpretation that the supervisor has distorted the 
information after receiving it from the subordinate* Unfortunately 
the approach presented by Road, Mellinger , and Maier, Hoffman, and 
Read fail to control for this plausible confounding interpretation 
and hence, loads to conclusions based on a questionable operational 
definition* 

7 
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Another criticism, and one which possibly casts the greatest 
degree of doubt on the validity of the disparity score, is that this 
operational definition of "upward distort ' on" fails to measure 
message sending behaviors. That is, conclusions about upward dis- 
tortion are put forth, yet these conclusions are in no way based 
directly upc.i massage sending behaviors of subordinates. At a 
minimum, c.w would hope thai an operational definition of "upward 
distortion" would capture the essence of this construct- i.e. , actual 
encoding behaviors of subordinates. Unfortunately the disparity 
score fails to capture this essence. It is an operational definition 
which leads to conclusions about a concept while distorting the 
crucial distinctions of that concept. Apparently,, Read, Melllnger, 
and Maier, Hoffman, and Read feel that upward distortion may be 
operatlonalized in a post-hoc fashion without ever measuring message 
sending behaviors per se\ If we accept the notion that a "good- 
operational definition should accurately map its referential ter- 
ritory, then an operational definition of "upward distortion" which 
falls to measure message sending behaviors is a misleading map. 

To Athanassiades' credit an operational definition was conceived 
which at least dealt solely with subordinates' behavior and which' 
was not reliant upon supervisors' perceptions. However, his oper- 
ational definition may be critiqued on three grounds. First, the 
substantive material of the supposed distortion dealt with a per- 
sonality description and not the "nuts and bolts" of mst organi- 
zational messages--!. e.s task related data. Hence, to accept 
Athanassiades' findings wo must also accept tho assumption that 
because subordinates will distort personal psychological profiles 



seen by superiors, they will also distort task related information, 
At best, this is a tenuous assumption* 

Second, Athanassiades , like Read, Mellinger, and Maier, Hoffman, 
and Read, is not analyzing subordinates' message sending behaviors* 
Rather, he is looking at the disparity of two sets of personality 
test scores^one anonymous and one open to the view of the supervisor* 
Athanassiades would have us believe that showing a supervisor a 
subordinate's score on a personality test is the same as having a 
subordinate send a message to the supervisor* Thus, while Athanassiade 
does eliminate the problems of supervisors 9 perception so inherent 
in the previously discussed studies, he does not eliminate the 
failure to measure message sending behavior* 

Third, Athanassiades 9 procedures are based on the assumption 
that when subordinates filled out the personality inventory 
anonymously their responses were accurate and undistorted; when 
the inventory was filled out for the supervisor's inspection the 
responses were distorted* Thus, in order for this disparity score 
to have any validity we must accept the assumption that the anonymous 
scores arm themselves accurate and valid indices of the subordinate's 
value system* Athanassiades glosses over this issue by stating^ 
"Gordon's questionnaires were used because tlwy are relatively 
short? self^administered, and are reported to have acceptable degrees 
of validity and reliability ; and because they are of the forced 
choice type which reduces fakeability of responses, and so provides 
seme additional assurance that the scores reveal relatively undistorted 
communi ca tion . " ^ However , in their discussion of Cordon's instru* 



ment, Robinson and Shaver observe that "The items arm transparent 
and may invite faking." 12 Furthermore, they stress that the "Cave At 
of the auth&r concerning validation must be kept in mind. In 
mhort* Athaaassiades* studies raise serious questions about the 
"relatively undistorted communication" in the anonymous condition* 

In summary f the disparity technique represented by Read, 
Hellinger, Kaier, Hoffman, and Read, and Athanassiades invite 
mlternativB interpretations of both the conclusions and the referen- 
tial nature of the underlying construct* 

Ques tJ. onnalre/ In tcrvi ew Responses 

This mpproach to operatlonallzing "upward distortion" is based 
upon the simple notion that a researcher interested in examining a 
given construct should ask subjects a series of questions related 
to that construct. Thus, a number of researchers interested in 
mzamining m&curacy of upward communication have collected data on 
fcha basis o£ responses to questionnaires and/or interviews . 

Representative studies using the questionnaire/interview approach 

1& 1 7 

include Ziwa, 14 Minter, 15 Burke and Wilcox, and Willits- 

Thm matjor criticism of any questionnaire/interview approach to 
opera ti onali zing "upward distortion" may be succinctly stated as 
follows: Data arm collected relative to what a subject says he 
does or believes, and not necessarily what he actually does or 
believes. Furthermore, Bince studies using the questionnaire/ 
interview mpproach rarely collect behavioral data etf validate self- 
report data we are forced to infer the former cm fcha* basis of the 
* latter. 
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For example, consider the following question taken from Likert's 
Organizational Climate questionnaire. This question deals with 
the variable "Accuracy of Upward Communication Via Line" and con- 
sists of an interval scale with four descriptors, The descriptors 
are as follows; 1) "Tends to be inaccurate?" 2) "Information that 
boss wants to hear flows} other information is restricted and 
filtered;" 3) "Information that boss wants to hear flows; other 

u 18 

information may be limited or cautiously given," and 4) "Accurate," 
Obviously i conclusions concerning responses to this item can only 
be couched in terms of what subordinates say about accuracy of 
upward communication. Any conclusion about upward distortion per se, 
on the basis of responses to Ms question is at best an inference. 

In short, questionnaire/inteririew data do not measure message 
Bending behaviors but rather meta message sending behaviors. That 
is, they measure messages about message sending behaviors* 

Selection and/or Transmission of Messages 

In reviewing the relevant literature only one study s found 

which operational! med "upward distortion" in terms of requiring a 

subject to select a message and transmit it to a supervisor, The 

study, conducted by O'Reilly arid Roberts, was designed to investi- 
ng 

gate ''selective filtration or omJ ision as a mechanism of distortion," 

In order to investigate this "mechanism" O'Keilly and Roberts 
designed m laboratory experiment manipulating direction of information 
flew (upward, downward, lateral), and quality of interpersonal 
relationship (hi trust, low trust}. The design required sub jeets ^ V ":?S 



to role play a given organizational member , select messages from a 
Bet of prepared messages* and then send the message (s) to either a 
peer, superior, or subordinate. The dependent measure cc. Isted of 
a frequency count of the number of messages transmitted which were 
pre-validated as important, unimportant, favorable to the subordinate, 
or unfavorable to the subordinate, 

h critical analysis of this approach to operational^ zing "upward 
distortion" results n the conclusion that the ingenuity and creativity 
manifested by Q'Reilly and Roberts have provided measures of precisely 
that which is measured"!, e* message selection and transmission. 
Unfortunately , since most subordinates create messages rather than 
select messages from a set of well defined messages, one is left 
with the unsettling feeling that this study operationaliz.es "mechanisms- 
of selective filtration or omission" without measuring encoding . ! 
behaviors per se. Obviously one could argue that the selection of* 
a prepared message may be viewed as one element of encoding behavior 
in that the procedure provides the subject with considerable lati- 
tude in deciding which message will be sent and which will not he 
sent. But again, the messages havt- not been created by the subject 
himself* Rather, they have beer created to fulfill various experi- 
mental conditions and designed to be representative of message 
types . 

Jfc should be noted, however, that some organizational positions 
do require the selection and transmission of messages and hence the 
O'Reilly and Roberts study is highly appropriate in generalizing to 
these positions* For example* traffic controllers and inventory controllers 



function primarily as informs -ion regulators* They select the data 
which will he sent, decide how it will be Bent, when it will be 
sent, and how much of it will be sent, In short, the O'Reilly and 
Uoberts study is extremely valuable if we focus primarily upon 
information processors who select rather than create messages* 

Actual Encoding of Messages 

Only three studies which operational! zed this construct in 

terms of manual encoding behaviors in organizational heirarchies 

20 

imrm found in reviewing the literature. Two of the studies were 
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ocnductmd by Kelley and Cohen with the former serving as a ■ 
m&dmi for the latter* 

Both ojf these studies were conducted in laboratory settings 
and both utilized similar procedures* In Kelley 's study subjects 
mrere assigned to mobility conditions (upward, downward) and/ status 
conditions (hi, low) on the basis of instructions provided by the 
mmperimentBE* In Cohen's study data were collected soley from 
subjects In the low (subordinate) status condition which was also 
Induced btf means of experimenter instructions* In both studies 
subjects were told that all aommunlcation with other groups in tfte 
hierarchy was to be done via written messages which would be 
relayed by the experimenter. A content analysis of these written 
wsmges pMOvided the data for both studies* 



Thm obvious strength of the operational procedures presented 
above is that upward distortion is measured on the basis of actual 
task related messages created by subordinates and sent to supervisors* 



Thus? when Kelley and Cohen conclude that upwardly mobile subcr** 
dinatcs tend to send massages to their supervisors which reflect 
favorably upon the subordinate, we Arm assured that this conclusion 
is based upon an analysis of actual messages sent by subordinates . 

Unfortunately , the data provided by Kelley and Cohen have been 
collected in the laboratory and not hierarchies of "real" organic 
matlons* Bence questions concerning ego involvement of Sujbjectg, 
interna lization of mobility manipulations f and artificiality of 
mxperimentml procedures may be raised leading to the superordinate 
question of external validity* Given these questions/ the conclusions 
provided by Kelley and Cohen are still considered " class ic&l" and 
included in most surveys of organlmationml communica tion • 

The third study utilizing actual encoding of messages was 
conducted by Krivonos** The data for his study were collected from 
participantm (subordinates) at the corporate headquarters of ^ 
large manufacturing companies* The subordinates were a&ke^ to" 
write messages to their supervisors based on simulated situations 
prepared for the study. The simulated situations were of m tagk 
m non-task nature and were either favorable or unfavorablm to the 
subordinate. Thus, each subordinate wrote four messages (counter- 
balanced to control for order effects) based on each of thm four 
stimulus situations; (1) task related/favorable; (2) tagk related/ 
unfavorable; (3) non- task related/ favorable; and (4) non~tmsk 
rela ted/unfavorable * 



Am with tAe studies by Kelley and Cohen, the strength of the 
procedures employed by Krxvonos is that distortion is measured on 
the basis of actual, messages created by subordinates , The advantage 
of the approach by Krivonos over that of Kelley and Cohen is that 
the study was conducted in the hierarchical structure of "real" 
organ! za tiom * 

Even though Krxvonos did take the innovative step of having 
actual encoding of messages in "real-life" hierarchies , his approach 
is still open to criticism, The major criticism of the Krivonos 
approach is that the messages were not actually sent to supervisors, 
but rather the subordinates were told to write the messages as if 
these messages would be sent to their supervisors,. Thus, not only 
were the situations from which the messages written simulated, but 
also, in a sense, the messages themselves were simulated 9 Therefore, 
while there was actual encoding behavior in this study, the message* 
sending behavior may have only been simulated rather than actual. 
Even with these criticisms in mind, Krivonos' is the first attempt 
to look at the encoding of messages in n real" organizational settings 

In Bummarizing the four operational approaches discussed in 
this paper we note that; (1) the disparity approach yields the 
most equivocal and misleading interpretations $ (2) questionnaire/ 
interview approaches yield meta message-sending data; (3) the 
laboratory approaches (actual encoding of messages and selection/ 
transmission of messages) provide data closely allignod with the 
referential nature of the construct, "upward distortion** but may 



be critiqued on grounds of external validity; and (4) the encoding 

of messages in actual organizational bettings may help to overcome 

this external validity problem, but this approach is composed of a 

Mingle study, which itself can be critiqued on the grounds of dealing 

with simulated rather than actual message-sending behavior* Moreover, 

of the thirteen studies reviewed, only Krivonos 9 demit with actual 

encoding behaviors in "rear* organizations, Athanasslades did deal 

with information "sent" in "real" organizations, but his studies 

examined distortion of psychological test profiles, not task related 

data* Thus, what we know about upward distortion is totally dependent 

on how we know it* And as argued thus far, how we know it is open 

to considerable criticism* 

-- . * 

Given the critical review thus far, the reader might ask? How 

than should we opera tionali ze upward distortion? The following 

section provides an answer to this question and in so doing stresses 

one major principle: fcfie data collected should measure the actual 

encoding behaviors of subordinates* 

Suggestions on Opera tionali zing 
Upward Distortion 

One method for operationalizing "upward distortion" is to 
provide the subordinate with a case {hypothetical or real) and then 
as* him to construct a message on the basis of that case Which would 
actually be sent to his immediate supervisor. This method, which ; 

"• - ' . • • ' ' -; - -* - '.' ; . • r*"~ •"- " T ■''*<• - . v 

is very similar to the approach used by Krivonos, not] only Raptures 
thm essence of the underlying ^iwtructS& .[^2 
hmhmviorm) , but also provides : an externally valid measure of accuracy . ■. • 



Xf the case represents the actual event, then anything the subordinate 

says about the evmt may be compared to the event with the comparison 

serving as a measure of degree of distortion. This technique is 

similar to analyzing rumor transmission as discussed by Allport and 
34 

Postman but is applied specifically to organizational hierarchies 
ms Krivonos did in his study. The difference between this approach 
and the one utilized by Krivonos is that this suggested method deals 
with actual rather than simulated message-sending behavior. 

A related approach is to use a modification of Davis 9 ECCO 
25 

analysis. Although primarily used to operationalize ttie concept 
^organizational grapevine, 1 * £CCQ analysis may be applied to 
analyzing the substantive changes in a message as it moves vertically 
in the hierarchy, Thus? subjects (supervisors and subordinates) 
would be instructed to record as accurately as possible the message 
as sent (by subordinate) or as received (by supervisor) , and the 
conditions under which it was sent or received (e,g, time of day, 
oral vs, written, etc.), It should also be noted, that this pro- 
oedure would provide a method for operatlonaliBlng March and Simon's 
concept? "uncertainty absorption," 26 a concept theoretically akin 
to upward distortion. 

Finally, upward distortion may be measured in a post-hoc 
fashion by analyzing the actual documents which organizational 
members utilize in carrying out their respective task functions. 
That is, researchers might consider analyzing memos, directives , 
mnd the sundry other written messages which subordinates send to 
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their supervisors during the eeurse of an average worA day. x„ a 
sense this procedure is a field setting application of the experi- 
mental procedures designed by Kelley and Cohen. 

For those skeptics who might argue that such an approach Is- 
impractical If -not impossible, we would suggest that they read two 
reports. The first is Adam' participant observer analysis of 
Vietnam related intelligence. 37 The second Is a report of an Inter- 
vention program conducted by Harvey and Beettger. 38 

Adams' analysis of the upward distortion in Vietnam related 
intelligence is based primarily upon- written documents sent from 
lower levels In the intelligence gathering structure to upper levels 
■ in the structure. His analysis of these documents dramatically 
illustrates the nature, .scope, and function of upward distortion. 

Harvey and Boettger also analyzed upward directed messages 
but conducted their analysis as outside intervention agents, 
essentially, their approach consisted of collecting written memos 
which were actually sent and used by a sub unit of an organization. 
After deleting all personal references in the memos, Harvey and 
Boettger used the edited memos as stimuli for group discussions 
concerning the general topic "communication improvement." 

Thus, actual written messages have been collected in field 
settings and since such memos constitute a significant portion of 
most subordinates' message sending options, they should continue 
to be collected and analyzed,, 



Conclusion 

The utility and necessity of operational definitions are 
obvious to anyone conducting research. , Just as obvious, however, 
is the realization that some operational definitions of a concept 
are "better" than other operational definitions of the same concept 
The "better" operational, definition is that which accurately maps 
its referential territory, highlighting the salient features, de- 
emphasizing the non salient features. 

In reviewing four different approaches to opera tionall zing 
"upward distortion" this paper has compared the more accurate maps 
with the less accurate maps. In addition, procedures were suggested 
to Improve the process wbereby we map the "upward distortion" 
territory. Since upward distortion is a central issue of organiza- 
tional communication it behooves us to utilize the best operational 

definitions as possible. To do otherwise is to distort the data 

of upward distortion. 
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